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A NOTE ON NILPOTENT REPRESENTATIONS 


MAXIME BERGERON AND LIOR SILBERMAN 

Abstract. Let T be a finitely generated nilpotent group and let G be a complex 
reductive algebraic group. The representation variety Hom(T, G) and the character 
variety Hom(r, G)//G each carry a natural topology, and we describe the topology of 
their connected components in terms of representations factoring through quotients 
of T by elements of its lower central series. 


1. Introduction 

Let G be the group of complex points of an affine algebraic group. When Y is 
a finitely generated group, one may parametrize the homomorphisms from Y to G 
by the images of a finite generating set. This realizes Hom(T, G) as an (affine) 
algebraic set, carved out of a finite product of copies of G by the relations of T. As a 
complex variety, Hom(T, G) admits a natural Hausdorff topology obtained from an 
embedding into affine space and it is easy to see (and well-known) that the analytic 
space structure on Hom(T, G) is independent of the chosen presentation of T. Here, 
we will only consider the case where G is reductive though, in principle, the questions 
we address below can be asked without this assumption. 

These spaces of homomorphisms are of classical interest (see Lubotzky-Magid 
HU and the references therein) and their algebraic topology has been the subject 
of much recent scrutiny (see, for instance, m si u m hb m usd, stemming in 
part from the work of Adem and Cohen [[QJ. In this context, it was recently shown 
by the first named author [6] that if T is nilpotent and K is a maximal compact 
subgroup of G, then there is a strong deformation retraction of Hom(T, G) onto 
Hom(T, K). This result was first established by homotopy-theoretic methods for T 
abelian by Pettet and Souto |THj and for T expanding nilpotent by Souto and the 
second named author. The result for arbitrary nilpotent groups was obtained in [6] 
by replacing these earlier approaches with algebro-geometric methods. Nevertheless, 
the machinery developed by Pettet-Souto and its followups is very well posed to 
the study of topological invariants. Accordingly, the goal of this note is to combine 
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these topological and algebro-geometric tools to obtain topological information about 
representation spaces of nilpotent groups. 

From now on, fix a non-abclian finitely generated s-step nilpotent group T. Recall 
that this means that the lower central series, defined inductively by 

r (i) = r > r (i+i) = [F> r (i)] 

has r (a) non-trivial but T( s +i) = {e}. The epimorphism T —* r/T(q induces an 
embedding 

Hom(r/r (i) ,G) -A Hom(r, G) 

which (for general groups T and G) is not even an open map. Nevertheless, we will 
show: 

Theorem 1.1. Let T be a finitely generated nilpotent group and let G be the group of 
complex points of a (possibly disconnected) reductive algebraic group. For all i > 2, 
the inclusion 

Hom(r/r (i) , G) 4 Hom(r, G) 

is a homotopy equivalence onto the union of those components of the target inter¬ 
secting the image of t. 

Consider Hom(r, G) as a based space by taking the trivial representation as the 
base point. In this case, Theorem 11.11 implies that the connected components 

Hom(r,G)i C Hom(r, G) and Hom(r/r (i) , G)i C Hom(r/r (i) , G) 

of the trivial representation are homotopy equivalent for all i > 2. Using this, we 
will describe the homotopy type of the component of the trivial representation in 
terms of abelian representations: 

Corollary 1.2. ForT and G as in Theorem o there is a homotopy equivalence 

Hom(T, G)i ~ Hom(Z rankffl ( r;Z ),G) 1 . 

To introduce the other space we study, note first that the action of G on itself 
by conjugation induces an action on Hom(T, G) and conjugate homomorphisms are 
often considered equivalent (this is the usual notion of equivalence of representations 
in GL n C). Accordingly one often wishes to understand the associated quotient but, 
unfortunately, the naive topological quotient is not a nice space: it need not even 
be Hausdorff. In order to “repair” this space, we use the affine geometric invariant 
theory quotient Hom(r,G)/G instead. This so-called character variety is usually 
endowed with the structure of an affine variety but, for our purposes, it may be 
constructed topologically as the universal quotient in the category of Hausdorff spaces 
(see Brion-Schwarz [10]). The systematic study of the topology of theses spaces has 
seen much recent development (see, for instance, 0 El EH HUES]). Concentrating 
on the component of the trivial representation, we will use Corollary 11.21 to prove: 
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Corollary 1.3. Let T be a finitely generated nilpotent group and let G be the group 
of complex points of a reductive algebraic group. Then 

(1) 7Ti (Hom(r, G)i) = 7Ti(G) rankHl ( r;Z \ and 

(2) 7tt ((Hom(r, G) // G) t ) = tt, (G/[G, G])™ kHl ( r ^. 

Corollary 1.4. Let G be the group of complex points of a connected reductive alge¬ 
braic group, let T C G be a maximal algebraic torus and let W be the Weyl group of 
G. IfT is a finitely generated nilpotent group and F is a field of characteristic 0 or 
relatively prime to the order of W, then: 

(1) H* (Hom(r ,G)r,F) 2* H*(G/T x r rank ^( r ^; F) w , and 

(2) H* ((Hom(r, G)//G) t ; F) = H*(T™ nkH ^ r ^; F) w . 

While the results above indicate many similarities between representation spaces 
of abelian and non-abelian nilpotent groups, the latter have a much richer topology 
than the former. For instance, recall that for a connected semisimple group S, the va¬ 
riety Hom(Z 2 ,S') is irreducible and thus connected [20]. Moreover, Hom(Z r , SL n C), 
Hom(Z r , Sp 2n C) and the corresponding character varieties are connected for all val¬ 
ues of r and n. The situation for non-abelian nilpotent groups is markedly different: 

Theorem 1.5. Let G be the group of complex points of a (possibly disconnected) 
reductive algebraic group. IfT is a finitely generated nilpotent group which surjects 
onto a finite non-abelian subgroup ofG, then Hom(r, G) and Hom(T, G)/G are both 
disconnected topological spaces. 

Since non-abelian free nilpotent groups and Heisenberg groups surject onto the 
non-abelian nilpotent group of order 8, this implies: 

Corollary 1.6. Let T be a non-abelian free nilpotent group or a Heisenberg group. 
If G is the group of complex points of a reductive algebraic group, then Hom(r, G ) 
and Hom(r,G)/G are connected if and only if G is an algebraic torus. 

Remark. All of the preceding statements remain true when G is replaced by a 
compact Lie group K. In fact, we will prove most of them in this setting before 
obtaining the complex reductive case via a homotopy equivalence. 

Outline of the paper. We begin Section [2] by describing compact representation 
spaces using a fibre bundle. Then, in Section [3] we use this bundle to prove Theorem 
11.11 and Theorem 11.51 along with their various corollaries. 

Acknowledgements. The authors would like to thank Alejandro Adem and 
Alexandra Pettet for their comments on a preliminary version of this note. The first 
named author would also like to thank Man Chuen Cheng, Justin Martel and Juan 
Souto for many stimulating conversations. 
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2. AN INTERESTING BUNDLE 

The goal of this section is to prove the following key proposition: 

Proposition 2.1. Let K be a (possibly disconnected) compact Lie group. IfT is an 
s-step nilpotent group with s >2, then the set of abelian groups 

J-:={p(r w ) C K:peRom(r,K)} 

admits a homogeneous manifold structure with finitely many connected components 
for which the projection map 

(1) V : Hom(T, A') -A T, p{p) = p(T (s) ) 

is a locally trivial fibre bundle. 

The proof of Proposition 12.11 relies on the following lemma: 

Lemma 2.2. For all m £ N there is an O = 0(m ) £ N such that, if N C SU m is an 
s-step nilpotent group with s > 2, then is an abelian subgroup of SU m of order 
bounded by O. 

Proof. Recall that is an abelian subgroup of SU m contained in the centre of N. 
As such, there is a direct sum decomposition C m = V\ © ... © V r and r characters 
Xi, ■■■ ,Xr '■ A'( s ) —» C x such that x.i 7 ^ Xj f° r all i ^ j and / y(v) = Xi(l) ' v f° r all 
7 G A( s ) and v G Xf. Moreover, for all 5 6 JV, 7 6 and v G Vi, we have 

7(g(v)) = ff(7( v )) = 9(Xt( 7) • v) = Xi( 7) • g(v). 

This allows us to consider the restrictions of the determinant homomorphism 

detj : N -A C x , d eti(g) := det(p|vj) 

where, since C x is abelian and s > 2, the subgroup must be contained in 
ker(deti). This means that for all i and all 7 £ iV( s ), we have 

det.( 7 ) = X<(lf mV ' = 1, 

so Xi(l) is always a root of unity of order bounded by m. Consequently, N^ s ) is 
conjugate in SU m to a subset of those diagonal matrices whose diagonal elements are 
roots of unity of order bounded by m. This completes the proof since the order of 
this finite set does not depend on s. □ 

Proof of Proposition 1 2. 1\ Choose a faithful embedding of K into SU m . By Lemma 
12.21 there is a constant O £ N uniformly bounding the order of abelian subgroups of 
K occurring as the image of T( s ) under homomorphisms p : T —> K. I 11 order to give 
T a homogeneous manifold structure, we first consider the slightly larger set 

T := {A C K : A is an abelian subgroup of order bounded by O}. 
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Observe that K° (the identity component of K ) acts by conjugation on T with 
closed stabilizers. As such, we can endow T with the orbifold structure with respect 
to which each Ji°-orbit is a connected homogeneous /l°-manifold (see [18]) - Con¬ 
cretely, if we define the “connected normalized’ as Nk°(H) := N K (H) fl K°, then the 
connected component of H G T is identified with K° / N k°(H). Having a topology 
on each J!°-orbit, we endow T with the disjoint union topology. Since K is a com¬ 
pact Lie group, there are only finitely many conjugacy classes of abelian subgroups 
of K of order bounded by O and, in particular, T has only finitely many connected 
components. 

A homomorphism p : T( s ) —> K need not extend to the full group T so the map 

P ■ Hom(r, K) T, p(p) = p(T {s) ) 

may not be surjective. Accordingly, we denote T := p(Hom(r,/l)) and observe 
by A^-equivariance of p that it is a union of connected components of T. Let 
Z C T denote the connected component of a finite abelian subgroup H G T and let 
V. := p~ 1 (Z) C Hom(r,/l). Since J- has only finitely many components, it follows 
that p is a continuous map and it now suffices to show that p: W, —> Z is a locally 
trivial fibre bundle. Observing once again that p is K°- equivariant, this follows at 
once from 0 Proposition 2.3.2]. More concretely, letting T-L(H) := p 1 (H), we can 
identify the restriction of p to "H with the twisted product 

(. K° x U(H))/N Ko (H) -> K°/N Ko (H) 

where Nk°(H ) acts on K° (resp. T-L(H)) by right multiplication (resp. conjugation). 

□ 


3. Proofs of the main results 

Let G be the group of complex points of a (possibly disconnected) reductive al¬ 
gebraic group and recall that such a G necessarily arises as the complexification of 
a (possibly disconnected) compact Lie group K. In this section, we use Proposition 
12.11 to prove the results mentioned in the introduction. In most cases, we prove a 
corresponding statement with K in lieu of G before obtaining the claimed result. 
We refer the reader to Onishchick-Vinberg [18] for basic facts about Lie groups and 
complex algebraic groups. 

Let T be an s-step nilpotent group with s > 2 and recall that, for all i, the 
epimorphism T —> r /Tm induces an embedding Hom(r/P(j), A') —> Hom(r, AT). 
Often, we shall abuse notation and identify Hom(r/r(q, A') with its image under 
this embedding. As a first consequence of Proposition 12 .II we obtain: 
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Proposition 3.1. Let K be a (possibly disconnected) compact Lie group. If Y is a 
finitely generated nilpotent group then, for all i > 2 , the inclusion 

Hom(r/r (i) , K) 4 Hom(r, K) 

is a homeomorphism onto the union of those components of the target intersecting 
the image of t. 

Proof. We proceed by induction on the nilpotence step of T. Recall from Proposition 
12.11 that 

P ; Hom(r, K) -A 7, p(p) = p(r (a) ) 

is a locally trivial bundle. If T is 2-step nilpotent, then the image of t consists of 
all representations factoring through the abelianization of T, that is those such that 
p(r (2) ) = {ex}- Since ex is fixed by the conjugation action of K, the subgroup 
{ex} G 7 is an isolated point in the given topology. Thus, for any p G p~ l (ex), the 
full connected component of p (which is path-connected) has trivial restriction to T ( 2 ) 
and we see that p~ l ({e^}) is the union of the connected components it intersects, 
completing the proof in this case. 

Suppose now that T is s-step nilpotent. If i — s, the same argument as for the 
base case applies. Otherwise, i < s and then 

r/r (() ^ (r/r w )/(r (() /r w ) 

where the nilpotence step of (T/Tp,)) is s — 1 . As such, the induction hypothesis 
implies that each of the following two embeddings 

Hom(r/r (i) , K) -A Hom(r/r (s) , K) -A Hom(r, K) 

is a homeomorphisms onto those components of the target intersecting its image and, 
consequently, that the same holds for their composition. □ 

Proof of Theorem [7771 The theorem follows at once by [ 6 ] Theorem I]. □ 

We can now prove: 

Corollary 3.2. IfT and K are as in Proposition 13.11 then there is a homeomorphism 

Hom(T, K)t = Rom(Z rankHl{r ' z) ,K) t . 

Proof. By Proposition 13.11 we have a homeomorphism 

Hom(r, K) t ^ Hom(i7 1 (r ; Z), K) t . 

Since HifT^Z) = r/[r, T] is a hnitely generated abelian group, we may identify 
Hi(T] Z) with Z r © A where r := rank PR(r; Z) and A is a finite abelian group. At 
this point we would like to show that Hom(Z r © A, K)% = Hom(Z r , K) t . Seeking 
a contradiction, suppose that p 0 G Hom(Z r © A, K) t maps A non-trivially into K. 
By assumption, there is a continuous path of representations [0,1] ha p t starting 
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at po and ending at the trivial representation p\ = 1. But now, this path induces 
a continuous deformation in Hom(A, K) of the representation p 0 U f° th e trivial 
representation. This is impossible since Lie groups contain no small subgroups. □ 

Proof of Corollary 11.21 The corollary follows at once by [6, Theorem I], □ 

Using this, we immediately obtain: 

Corollary 11.31 Let G be the group of complex points of a reductive algebraic group. 
IfT is a finitely generated nilpotent group, then: 

(1) 7Ti(Hom(T, G)i) = TTi(G) rankHl( - T ’ z \ and 

(2) 7r 1 ((Hom(T, G)//G)\) = 7T 1 (G/[G J G}) rankH ^ T ’ z \ 

Proof. The two formulas follow at once from Corollary 11.21 by the main results of 
Gomez-Pettet-Souto |H>] and Biswas-Lawton-Ramras [8]. □ 

In order to prove our second corollary, we need the following: 


Lemma 3.3. If K is a compact Lie group and T is a finitely generated nilpotent 
group, then Hom(T, K)t/K = (Hom(T, K)/K) t . In particular, Hom(T, K) is con¬ 
nected if and only i/Hom(T, K)/K is connected. 

Proof. Recall from Corollary 13.21 that any p £ Hom(T,/l)i factors through the tor¬ 
sion free part of L/i(T;Z). As such, by [5J, Lemma 4.2], p £ Hom(T,/l) 1 if and only 
if there is a torus T C K such that p(T) C T. Since this property is preserved under 
conjugation by elements of K , it follows that (Hom(T, K)/K )i coincides with the 
quotient Horn(T, K) t /K. □ 


We can now prove the cohomological formulas mentioned in the introduction. 


Corollary 11.41 Let G be the group of complex points of a connected reductive alge¬ 
braic group, let T C G be a maximal algebraic torus and let W be the Weyl group of 
G. IfT is a finitely generated nilpotent group and F is a field of characteristic 0 or 
relatively prime to the order of W, then: 


(1) Lf*(Hom(T, G)i; F) 2* H*(G/T x T rankH ^)- F ) w , and 

(2) iL*((Hom(T, G)//G)r, F) = H*(T rankH ^ r ^-, F) w . 


Proof of Corollary \1.4\ Following Pettet-Souto [T9, Corollary 1.5], let K cGbea 
maximal compact subgroup such that Tk '■= T D K is a maximal torus in K. Notice 
that, for any r £ N, 

K/T k xTa G/T x T r 

is a lU-equivariant homotopy equivalence and, in particular, that 


( 2 ) 


H*(K/T k xTf 


H*{G/T x T r ) w . 
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Here, it follows from Baird [5J Theorem 4.3] that the left hand side of the equation is 
isomorphic to R*(Hom(Z r , K )n). Now, letting r := rank//’ 1 (T; Z), our first formula 
follows at once from the homotopy equivalences 

Hom(r,G)i ~ Hom(Z r ,G) 1 ~ Hom(Z \K) t 

provided by Corollary 11.21 and [6] Theorem I], Finally, it is also due to Baird [5j 
Remark 4] that Hom(Z r , K)t/K = Tff/W so our second formula follows from the 
homotopy equivalence and homeomorphisms 

(Hom(T, G)HG)j ~ (Hom(T, K)/K)% = Horn (T,K) t /K = Hom(Z r ,K) 1 /K. 

provided by [BJ Theorem II], Lemma [3.31 and Corollary 13.21 □ 

Remark. The homotopy types of distinct components of representation spaces are 
typically different. For instance, if we take T to be the discrete Heisenberg group 
H 3 ( Z), then Hom(r, SL 2 C) decomposes into a simply-connected component and a 
non simply-connected component. In fact, this phenomenon already occurs for T 
abelian as illustrated in Gomez-Adem [2] and Gomez-Pettet-Souto [15] . 

Theorem 11.51 Let G be the group of complex points of a reductive algebraic group. 
If T is a finitely generated nilpotent group which surjects onto a finite non-abelian 
subgroup ofG, then Hom(r,G) and Hom(r, G)//G are both disconnected. 

Proof. Let if : Y —» N be a surjective homomorphism onto a finite non-abelian 
subgroup of G and let K be a maximal compact subgroup of G containing N. Notice 
in particular that if E Hom(r, K ) C Hom(r,G). Since Hom(r, K) ~ Hom(r,G) 
and Hom(r, K)/K ~ Hom(r, G)//G by [ 6J , and since Hom(r,/l) is disconnected if 
and only if Hom(r, K)/K is disconnected by Lemma 13.31 it suffices to prove that 
Hom(r, K ) is disconnected. 

Seeking a contradiction, suppose that Hom(r, K ) is connected and recall from 
Proposition 13.11 that, in this case, Hom(r/P(n, K) is connected for all i > 2. Choose 
a minimal s G N with the property that ^(r^+i)) = ex and denote the s-step 
nilpotent group r/T( s+1 ) by f. If we consider the fibre bundle (c.f. Proposition 12.ip 

P ■ Hom(f, K) -E 7 , p(p) = p( f (s) ), 

then piijf) = if(T( s )) ^ ex- As such, by Proposition 13.11 and our assumptions, 

if £ Hom(f, K) t = Hom(r/r (s+ i), K)t ^ Hom(r, K) t = Horn (r, K) 

and this contradiction completes the proof. □ 

Corollary 11.61 Let r be a non-abelian free nilpotent group or a Heisenberg group. 
If G is the group of complex points of a reductive algebraic group, then Hom(r, G ) 
and Hom(r,G)/G are connected if and only if G is an algebraic torus. 
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Proof. If G is disconnected or not simply-connected then [19i Corollary 1.3], [6J Theo¬ 
rems I and II] and Lemma [373] show that Horn (/A (T; Z), G) and Hom(iy 1 (T; Z), G)//G 
are disconnected. As such, it suffices to consider the case where G is simply- 
connected. Notice that such a G contains a subgroup isomorphic to SL 2 C and, 
since SL 2 C contains a copy of the non-abelian group Q of order 8 generated by the 
matrices 

( 0 -i ) and ( -1 0 ) ’ 

so does G. Since Q is a Z/2Z central extension of Z/2Z x Z/2Z, it follows that if 
T is either a non-abelian free nilpotent group or a Heisenberg group, then T surjects 
onto Q. The claim now follows from Theorem 11.51 □ 
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